Measuring The Report Card : The Validity Of Pay - For - Performance Metrics In Timothy Ferris

To assess the validity of performance measures used in a nationwide pay-forperformance (P4P) project on hip and knee replacement, we analyzed hospital performance data from a Medicare P4P initiative and compared them to publicly available outcomes data. Overall, the ability to measure hospital quality was poor. A hospital’s ranking on the composite score was primarily determined by process measures. A higher composite quality score was not associated with lower rates of complications or mortality. The current Medicare P4P quality measure has limited validity because of poor discrimination, lack of measure balance, and lack of correlation with important clinical outcomes. [Health Affairs 28, no. 2 (2009): 526–532; 10.1377/hlthaff.28.2.526] P ayf o r p e r f o rmanc e (P4P) programs have been advocated as a method to improve the quality of health care in the United States.1 The success of P4P programs hinges upon the development of performance measures that accurately reflect the quality of clinical care delivered. Ideally, quality measures should be easily obtainable, discriminate between highand low-quality providers, be adjusted for the severity of casemix, and correlate with important external measures such as mortality and complication rates.2 Although numerous P4P programs have been implemented, little is known about how well the quality measures used by these programs actually perform, especially in the resource-intensive surgical fields.3 Rigorous assessment of performance measures should precede expansion of P4P programs. Suitability of arthroplasty for P4P. P4P programs target total hip and knee replacement because arthroplasty is one of the few well-defined procedures in medicine where performance can be measured. The surgical procedure is relatively standardized. Existing treatment care guidelines and clinical pathways facilitate the development of clinical performance measures.4 Outcomes after total hip and total knee arthroplasty are well studied, and the relationship between greater hospital case volume and improved outcomes has been described. Total joint replacement is 5 2 6 M a r c h / A p r i l 2 0 0 9 H e a l t h T r a c k i n g DOI 10.1377/hlthaff.28.2.526 ©2009 Project HOPE–The People-to-People Health Foundation, Inc. Timothy Bhattacharyya ([email protected]) is chief of orthopedic trauma at Suburban Hospital in Bethesda, Maryland. Andrew Freiberg is chief of arthroplasty at the Massachusetts General Hospital in Boston. Priyesh Mehta is a medical student at the University of New England College of Osteopathic Medicine in Biddeford, Maine. Jeffrey Katz is an associate professor in rheumatology at Brigham and Women’s Hospital in Boston. Timothy Ferris is medical director at the Mass General Physicians Organization. on A ril 7, 2017 by H W T am H ealth A fairs by http://conealthaffairs.org/ D ow nladed fom common and costly—more than $9 billion per year in the United States.5 Together, these factors make arthroplasty an attractive candidate for P4P. Medicare P4P project. The Centers for Medicare and Medicaid Services (CMS)/Premier Hospital Quality Initiative Demonstration (HQID) was a voluntary demonstration project in which hospitals reported data on quality and outcomes.6 The CMS and Premier Inc., an organization owned by not-for-profit hospitals, collaborated in July 2003 to launch HQID. This became the first national P4P demonstration to examine the relationship between quality and cost. The HQID included 260 hospitals in 38 states, focusing on five primary clinical areas: acute myocardial infarction (AMI), heart failure, community-acquired pneumonia, coronary artery bypass graft (CABG), and hip and knee surgery. The CMS began collecting data in October 2003 and released its data from year one in 2005. Through the Deficit Reduction Act (DRA) of 2005, Congress authorized the development of a Medicare hospital value– based purchasing program. On 21 November 2007 the CMS released its data from year two and submitted a plan to Congress to implement P4P on a national scale in 2009. Congress approved the plan and will be responsible for creating a final program. In the first year of the demonstration project, hospitals that scored in the top 10 percent on the composite quality measure received a performance bonus consisting of 2 percent of diagnosis-related group (DRG) payments for total hip and knee arthroplasty for the study year. Hospitals in the second decile of the composite quality measure received a 1 percent DRG bonus. All hospitals scoring in the top 50 percent of performance were publicly recognized on the HQID Web site. Medicare quality measure. The CMS has proposed one method to measure the quality of hip and knee replacement at the hospital level.7 It includes a composite score created from three measures of surgical process quality and three measures of surgical outcome. The composite score was used in the HQID. Study Methods Hospital performance assessment. Using publicly available data, we performed a cross-sectional analysis of hospitals participating in the hip and knee segment of the HQID and assessed the validity of the CMS quality scores with clinically important outcomes.8 We assessed hospital performance in two ways: hospital performance tier and composite quality index. First, we identified four ordinal tiers of hospitals. Tier 1 hospitals were in the top 10 percent of performance, tier 2 hospitals were in the second decile, tier 3 hospitals were in the top 50 percent but not in the top two deciles, and tier 4 hospitals were in the bottom 50 percent. Second, we calculated the hip and knee composite quality index according to CMS guidelines. Correlating quality measures with outcomes. We obtained three external measures of hospital performance. First, we collected data on inpatient mortality after hip and knee arthroplasty. Second, as a measure of complications, we used “iatrogenic complications” (physician-caused complications) and “urinary tract infection” (UTI), which are riskadjusted, validated measures in surgical patients.9 Third, we compared the performance measures to surgical volume (total number of total hip and knee arthroplasties performed at a hospital in the study year), because volume is known to correlate with quality.10